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ABSTRACT 



Multiple data stream searching is achieved by encoding a 
video broadcast into multiple types of data streams including 
video, audio and subtitle data. Addresses axe generated for 
each of the data streams pointing to a position of the data 
stream to be stored on a record medium. These addresses are 
stored in areas of the record medium reserved by type of data 
stream such that location of any event represented by data in 
a data stream can be achieved by locating the data streams 
of the same type as that of the data to be located. A 
computer-readable memory directs a computer to store these 
addresses and to decode data streams at these addresses 
which correspond to the type of data stream to be searched. 
The data streams which are of the type to be searched are 
decoded and the event to be searched is found in these 
decoded data streams. 

22 Claims, 20 Drawing Sheets 
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1 

ENCODING AND DECODING OF DATA 
STREAMS OF MULTIPLE TYPE S 
INCLUDING VIDEO, AUDIO AND SUBTITLE 
DATA AND SEARCHING THEREFOR 

BACKGROUND OF TOE INVENTION 

The present invention relates to searching subtitles and, 
more particularly, to searching multiple kinds of subtitle data 
streams. 

Television broadcasting or video reproduction (such as 
from a video disk) provides subtitles superimposed on the 
video image. Problematically, the subtitles are permanently 
combined with the underlying video image and cannot be 
manipulated at the receiving (or reproducing) end. The 
subtitles, for example, cannot be searched for information 
concerning a specific scene occurring in the video image or 
sound in its corresponding audio track. 

Compact Disc Graphics (CD-G) provide some flexibility 
in searching subtitles because this technique records graph- 
ics in the form of subcodes. However, CD-G has a serious 
disadvantage because this technique is limited to compact 
disc (CD) applications, which are slow by television stan- 
dards. That is, the CD-G technique does not lend itself to 
manipulation of subtides in real-time television broadcasts 
or video reproductions. 

As will be shown with reference to FIGS. 18A-C and 19, 
the lead time required to generate a full CD-G screen is 
grossly inadequate for normal television or video broad- 
casts. FIG. 18A depicts the CD-G data format in which one 
frame includes 1 byte of a subcode and 32 bytes of audio 
channel data, 24 bytes of which are allocated for L and R 
audio channel data (each channel having 6 samples with 2 
bytes per sample) and 8 bytes allocated to an error correction 
code. The frames are grouped as a block of 98 frames 
(Frame 0, Frame 1, . . . , Frame 96 and Frame 97) as shown 
in FIG. 18B and eight of these blocks P,Q,R,S,T,U,V and W 
are transmitted as shown in FIG. 18C. The subcodes for 
Frames 0 and 1 in each block are reserved for sync patterns 
SO, SI, whereas the subcodes for the remaining 96 frames 
are reserved for various subcode data. The first two blocks 
P, Q are allocated to search data employed for searching 
through record tracks, while the remaining 6 blocks R,S,T, 
U,V and W available for graphic data. 

CD-G transmits each block of 98 frames at a repeating 
frequency of 75 Hz. Thus, the data transmission rate for 1 
block is (75 Hzx98 bytes) 7.35 kHz, resulting in a subcode 
bit rate of 7.35 K bytes/s. 

The transmission format for transmitting the information 
present in blocks R,S,T,U,V and W is shown in FIG. 19, 
wherein each of the 96 frames (2,3, ... 97) of the 6 blocks 
(R,S,T,U,V and W) are transmitted as a packet including 6 
channels (R to W) of 96 symbols per channel (a symbol 
comprising three cells). The packet is further subdivided into 
4 packs of 24 symbols apiece (symbol 0 to symbol 23), with 
each pack storing a CD-G character. It will be appreciated 
that, a CD-G character is made up of 6x12 pixels and, 
therefore, is easily accommodated in each 6x24 pack. 
According to the CD-G format, the 6x12 CD-G character is 
stored in the six channels of (R,S,T,U,V and W) at symbols 
8 to 19 (12 symbols). The remainder of the symbols in each 
of the packs store information about the character 

Mode information is one example of information stored in 
the packs and is stored in the first 3 channels (R, S, T) of 
symbol 0 in each pack. Item information is another example 
which is stored in the last 3 channels (U, V, W) of symbol 
0. A combination of the mode information and the item 



,861 

2 

information defines the mode for the characters stored in the 
corresponding pack as follows: 



TABLE 1 



5 



10 



Mode 


[tern 




000 


000 


0 mode 


001 


000 


graphics mode 


001 


001 


TV-graphics mode 


111 


000 


user's mode 



An instruction is another example of information stored in 
the packs and is stored in all of the channels of symbol 1. 
Corresponding mode, item, parity or additional information 

15 for the instruction is stored in all of the channels of symbols 
2 to 7. Parity information for all of the data in the channels 
of symbols 0 to i 9 is stored in all of the channels of the last 
4 symbols (symbols 20 to 23) of each pack. 

As discussed, the CD-G system is slow. The CD-G data 

20 is transmitted at a repeating frequency of 75 Hz and, 
therefore, a packet which contains 4 packs is transmitted at 
a rate of 300 packs per second (75 Hzx4 packs). That is, with 
1 character allocated to the range of 6x12 pixels, 300 
characters can be transmitted in 1 second. However, a CD-G 

25 screen is defined as 288 horizontal picture elementsxl92 
CD-G vertical picture elements and requires more than twice 
the 300 characters transmitted in 1 second. The total trans- 
mission time for a 288x192 screen is 2.56 seconds as shown 
by the following equation: 

30 

C28#6)x(192/12)-r300-2.56 seconds 

With the CD-G system, searching for a specific event 
would be extremely time consuming because the time to 
regenerate each screen (2.56 seconds) by itself is extremely 

35 long, when it is considered that screens are usually refreshed 
in tenths of a second. This problem is compounded when 
hexadecimal codes are used for the characters because each 
hexadecimal expression requires 4 bits to represent 1 pixel. 
As a result, 4 times the data described above is transmitted, 

40 thereby increasing the transmission rate to 10.24 seconds 
(4x2.56 seconds). Since each screen requires a sluggish 
10.24 seconds for transmissions a continual transmission of 
screens means that a lag time of 10.24 seconds is experi- 
enced when transmitting screens using the CD-G technique. 

45 In one type of system (known as the CAPTAIN system), 
dot patterns, as well as character codes, represent the sub- 
titles. This system, however, does not appear to be signifi- 
cantly better than the CD-G system and suffers from some 
of the same disadvantages. That is, both systems lack the 

50 capability to search for a specific event efiBciently. In 
addition, these systems do not provide subtitles with suffi- 
cient resolution power in displaying the subtides. The CD-G 
system designates only 1 bit for each pixel, and this binary 
pixal data creates undesired aliasing and nicker. The CAP- 

55 TAIN system, for example, is developed for a 248 
(horizontal picture elements) by 192 (vertical picture 
elements) display, i.e., a low resolution display, and not for 
high resolution video pictures of 720x480. 

OBJECTS OF THE INVENTION AND 
SUMMARY OF THE INVENTION 

An object of the invention, therefore, is to provide an 
encoding method and apparatus for encoding multiple types 
of data streams to be searched. 
65 Another object of the invention is to provide an encoding 
technique that enables a user to search for a particular 
substitle. 
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A further object of the invention is to provide a computer- FIGS. 18 A to 18C depict the arrangement of data accord- 
readable memory for directing a computer to search the ing to a CD-G format; and 

multiple types of data streams. FIG. 19 depicts a transmission format of the data in the 

An even further object of the invention is to provide a CD-G format, 

decoding method and apparatus for decoding multiple types 5 

of data streams to be searched DETAILED DESCRIPTION OF THE 

In accordance with the above objectives, the present PREFERRED EMBODIMENTS 

invention provides an encoding apparatus and method which Referring now to the drawings, wherein like reference 

encodes multiple types of data streams such that an event in numerals designate identical or corresponding parts 

the video broadcast represented by data in the data streams 10 throughout, the present invention will be described, 
can be located by locating data streams of the same type as 

the data. Decoding Apparatus 

The present invention further provides a computer- ^ ^ decodm shown m nG x ^ 

readable memory for directing a computer to search for an ^ ^ t and ^ tQ decodc a 

event in the video broadcast represented by data in the data ^ ^ ^ A u of ^ da(a 

streams by searching data streams of the same type as the causes me reproduced signal to be processed and 

data to be searched. seQt ^ a decoder 7. The system controller commu- 

The present invention further provides a decoding appa- n i ca tes with a controller 35 (FIG. 2) of the subtitle decoder 

ratus and method which decodes multiple types of data 2Q t0 decode the Cities and superimpose them onto a 

streams such that an event in the video broadcast represented decoded video image for display on a television screen, 

by data in the data streams can be searched by searching data j jj i 

7 . . 4 j * * u u j A. data decoder and demultiplexer 1 receives a digital 

streams of the same type as the data to be searched. . , , , f , \rr>n -pu j * 

signal reproduced from, for example, a VCR. The data 

BRIEF DESCRIPTION OF THE DRAWINGS decoder and demultiplexer error decodes the reproduced 

. . , . . 25 signal, preferably employing an Error Correcting Code 

A more complete appreciation of the present invention (Ecp) technique> and demultiplexes the error decoded 

and many of its attendant advantages will be readily signal int0 video , and audio data 

obtained by reference to the following detailed description sUeams A me % ^ ^ for e ^ ^ a 5uffer 

considered in connection with the accompanying drawings, memory asaworkarea for me purpose 0 f error decoding and 

in which: 30 demultiplexing the reproduced signal 

FIG. 1 is a block diagram of a data decoding apparatus of A ^ 3 ^ demultiplexed video data 

the present invention; from a Wdeo daU stream A mem0 ry 4 may be employed for 

FIG. 2 is a block diagram of the subtitle decoder depicted ^ O pe ration of decoding the video data similar to the 

in FIG. 1; operation of the memory 2 employed with data decoder and 

FIGS. 3A and 3B are tables of addresses according to the 35 demultiplexer 1. 

present invention; ... A letter box circuit 5 converts the decoded video data with 

FIGS. 4A and 4B is a diagram depicting subtitle search a 4-3 aspect ratio to a 16:9 aspect ratio. The conversion is 

operation in normal and trick playback modes; performed using a 4 to 3 decimation process, whereby every 

FIG. 5 is a table of communications between the system ^ four horizontal lines are decimated to three horizontal lines, 

controller of FIG. 1 and the controller of FIG. 2; thus squeezing the video picture into a 2 A picture. According 

FIG. 6 is a table of parameters for the communications to the letter box format, a vertical resolution component is 

between components of FIG. 1 and FIG. 2; derived from the remaining Ya of the video picture which is 

FIGS. 7A to 7C are signal diagrams demonstrating data employed to enhance the vertical resolution of the decimated 

encoding of the present invention; 45 video P icturc - A timia g adjustment memory 6 times the 

FIG. 8 is a color look up table referred to when encoding transmission of the video picture to ensure that the % of the 

subtitle data • er picture is not transmitted . When the decoded video 

^1^,0 n.1 « ATI ^ i_, 1 * • c data generated by the video decoder 3 is already in a 16:9 

FIGS. 9A and 9B constitute a block diagram of the ^ ^ ^ ^ ^ ^ ^ g ^ ded _ 

encoding apparatus of the present invention; m ^ Qn ^ ^ ^ ^ ^ dala direcUy 

FIGS. 10A and 10B depict a block diagram for the wipe 50 ^ subtitle decoder 7. 

data sampler of FIG. 9A; The decoded subtitle data demultiplexed by the data 

FIG. 11 is a color look up table referred to when con- decoder and demultiplexer 1 is sent directly to subtitle 

ducting a color wipe operation; decoder 7 which decodes the subtitle data according to 

FIG. 12 is a graph for the explanation of a code buffer ss instructions from system controller 14 and mixes the 

operation; decoded subtitle data with the decoded video data. 

FIG. 13 is a block diagram describing the internal opera- A composite encoder 8 encodes the mixed subtitle data 

lion of the code buffer in FIG. 2; an d video data into a suitable video picture format, such as 

FIGS. 14A to 14C depict a scheme for the colorwiping NTSC, PAL or the like. A mode display 9 interfaces with a 

operation; 50 user ^6 indicates, for example, the mode of a television 

FIG. 15 is a block diagram depicting the colorwiping monitor connected to the illustrated apparatus. A D/A con- 
operation according to FIGS. 14A to 14C; verter 10 converts the encoded signal received from the 

FIGS. 16 A to 16C depict a scheme for the dynamic composite encoder into an analog signal suitable for display 

positioning operation; in the indicated mode, such as NTSC or PAL. 

FIG. 17 is a block diagram depicting a circuit for the 65 The audio portion of the audio/video signal decoded by 

dynamic positioning operation according to FIGS. 16A to the data decoder and demultiplexer 1 is decoded by an audio 

16C; decoder U which decodes the demultiplexed audio data 
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using a memory 12, for example. The decoded audio data not overflow the code buffer. The scheduler controls read/ 

output from the audio decoder is converted into an analog write access to the code buffer by determining the bandwidth 

audio signal appropriate for reproduction by a television for an I/O port (not shown) which receives the bit streams 

monitor by a D/A converter 13. selected by tbe word detector. The bandwidth refers to the 

5 read/write rate and is calculated by dividing the rate at which 

Subtitle Decoder the demultiplexer demultiplexes data by the number of 

parallel bits written or read from the code buffer. For 

Subtitle decoder 7, as will be discussed with reference to example, a data rate from the demultiplexer of 20 Mbps 

FIG. 2, decodes the encoded subtitle data and mixes the divided by 8 parallel bits results in a 25 Mbps rate of data 

decoded subtitle data with the appropriate video data. Con- read from the code buffer. Therefore, the scheduler will set 

trailer 35 (FIG. 2) controls the operations of the subtitle 1 me read/write rate of the I/O port in order to maintain a 

decoder and communicates with the system controller 14 of consistent flow rate of data into and out of the code buffer, 

the decoder (FIG. 1) using the command signals shown in The code buffer, thus, receives the subtide data (Bitmap) and 

FIG. 2 (as listed in FIG. 5). Together, the controller 35 and awaits a decode start signal from the system controller to 

system controller 14 time the decoding of the subtitle data so rc ad out mc d aia . 

that the subtide data is mixed with video image data at the 15 Advantageously, the system controller executes reading in 

proper position whereat the subtities arc to appear on the reaJ time wheQ it ^ determined fr om mc horizontal and 

video image. vertical sync signals that the television display is at a 

A word detector 20 of the subtide decoder receives the position corresponding to the position indicated by the 

subtide data in groups of bit streams reproduced from a disk, 2Q position data. For real time display, the reading rate should 

the bit streams being stored on the disk in packets. Each correspond to a picture element sampling rate, preferably 

group of bit streams makes up one frame (or page) of 13.5 MHz. As discussed, the subtitle data preferably is 

subtitles to be superimposed on a video image. Different written into the code buffer at a rate of 2.5 MHz or more, 

groups of bit streams may represent subtitles displayed in Thus, the 133 MHz sampling clock is divided into four 

different playback modes, such as normal playback, fast- 25 clock cycles of 3375 MHz each. One of these 3.375 MHz 

reverse or fast-forward, alternatively referred to as trick clock cycles is allocated to writing (because writing requires 

modes. The system controller indicates to the word detector at least 2.5 MHz) and the remaining three clock cycles are 

using a stream_select signal which playback mode is to be allocated to reading data from the code buffer, thus satisfy- 

adopted for display and the word detector selects the appro- ing the requirement for real time display, 

priate bit stream of signals for the indicated playback mode. 3Q The read/write operation described is not only advanta- 

In the case where different video images are displayed on geously performed in real time, but also provides high 

different channels, the system controller indicates the appro- resolution. Eight bits of the subtide data are read from the 

priate channel to the word detector correspondingly in a co d e buffer 22 for each of three read clock cycles, or 

ch_select signal and the word detector changes channels to twenty-four bits per sampling clock. When display of the 

receive only those bit streams on the selected channel. 3S picture is conducted by the television monitor every fourth 

A group of bit streams making up one frame and received clock cycle, one-fourth of the twenty-four bits, (24/4 =) 6 

by the word detector includes header information (s. header) bits are displayed at every clock cycle. That is, each subtide 

which describes the format of the group of bit streams. The picture element may comprise six bits, which is more than 

header information is accompanied by header error infor- sufficient to achieve a high quality of resolution for the 
mation (header error) and data error information (data error). ^ subtides. 

The system controller uses the header information to deter- The operation of the code buffer 22 and corresponding 
mine how to parse the group of bit streams and extract the components of FIG. 2 is depicted in the block diagram in 
relevant subtitle data therefrom. The system controller uses FIG. 13. The code buffer 22-1 accumulates bit streams of 
the header error information to correct anomalies in the subtide data until at least one page of subtide data is 
header information and uses the data error information to 4S accumulated in the code buffer. The subtitle data for one 
correct anomalies in the subtitle data. page ^ transferred from the code buffer 22-1 to the display 
The word detector forwards the subtide data (Bitmap) memory 22-2 (which acts as a buffer for the subtitle decoder) 
along with other decoded information (including a presen- when the subtide portion of the display time stamp (PTS) is 
tation time stamp PTS, position data position_data and aligned with the synchronizing clock (SCR). The synchro- 
color look up table data CLUT„data) to a code buffer 22. 50 nizing clock advances a pointer in the display memory 22-2 
The PTS is a signal that indicates the precise time when the during reading indicating which address of the stored sub- 
audio, video and subtitle data for a frame is transmitted so tide data is being currently read. It will be noted that placing 
that the system controller knows when to demultiplex the the code buffer and display memory in a single unit is 
data from the reproduced signal. The position data indicates preferred since the code buffer need only increment one 
the horizontal and vertical position where the subtitles are to 55 pointer for pointing to the current address in the display 
be superimposed on the video image. The CLUT_data memory 22-2 which stores the next set of subtide data. With 
indicates which colors are to be used for the pixels making an internal memory, therefore, virtually no delay is attrib- 
up the subtitles. For example, the system controller 14 uted to a transfer operation, resulting in a high speed transfer 
determines that a video image is being displayed and sends of the subtide data. 

the subtide data to subtitle decoder 7 at the time indicated by 50 When the code buffer is read during a normal playback 
the time stamp (PTS) and causes the subtide decoder to mode, the synchronizing clock advances the pointer of the 
output the corresponding subtide data (Bitmap) at a position display memory 22-2 at each clock pulse. However, during 
in the video image represented by the horizontal and vertical special (or trick) reproduction (such as fast-forward, fast- 
position indicated by the position_data in the color indi- reverse playback modes), the pointer is advanced at a 
cated by the CLUT_data. 65 different rate. To this end, a special command is first sent to 
Ascheduler 21 is provided to ensure that the data received the controller 35 and the controller sends back an acknowl- 
by the code buffer 22 from the demultiplexer 1 (FIG. 1) does edge signal (special_ack), acknowledging that special 
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reproduction is to be initiated. To uniformly speed up (or 
slow down) the operations of the subtitle decoder according 
to the special reproduction rate, the system clock reference 
(SCR) can be altered by adding or subtracting clock pulses. 
Subtraction pulses are created at an n times rate correspond- 5 
ing to the rate of fast-feeding or fast-reverse feeding. For 
example, at the time when special reproduction is 
commenced, real time subtraction is performed on the bit 
stream of subtitle data read out from the code buffer at the 
n times rate and the pointer advances at the desired rate to 1Q 
effect the special playback mode. When the special repro- 
duction operation corresponds to a pause operation, on the 
other hand, no subtraction pulses are created. Instead, an 
identical frame is continuously read from the code buffer 
repeatedly, thus providing the illusion sensation that the 
subtitles are paused. 

The reading operation is ended when subtitle decoder 7 
determines that an end of page (EOP) of the subtitle frame 
is reached. The system controller sends a repeat time signal 
to the controller 35 which indicates the length of a page. An 2 q 
inverse run-length circuit 24 includes a counter and sends a 
display end signal to the controller 35 when the count value 
of the counter reaches the value indicated by the repeat time 
signal. When controller 35 determines that the repeat time is 
reached, the reading operation of the code buffer is stopped. ^ 
For purposes of this invention, the code buffer preferably 
stores at least two pages of subtitle data because one page 
will be read while another page is written into the code 
buffer. 

Controller 35 issues a buffer overflow signal to system 30 
controller 14 when an overflow of code buffer 22 occurs. An 
overflow can be determined when the controller receives the 
display end signal from inverse run-length circuit 24 before 
word detector 20 receives an end of page (EOP) signal on 
the following page. At that time, the system controller 35 
withholds transfer of subtitle data from data decoder and 
demultiplexer 1 (FIG. 1) to the word detector to prevent an 
overflow of the code buffer. When an overflow condition has 
passed, the next stream will be written into the code buffer 
and displayed at the correct display start position. 40 

An underflow condition exists when code buffer 22 has 
completed reading the subtitle data for an entire page and no 
further data exists in the code buffer. The code buffer is 
depicted with a capacity of two pages by the "code buffer 
size" line in FIG. 12. Graphically, an underflow would 45 
appear in FIG. 12 as one of the vertical portions of line (Q 
which extends below the lower limit of the code buffer. By 
contrast, an overflow condition is graphically depicted in 
FIG. 12 when the subtitle data read into the code buffer is too 
large, i.e., the horizontal portion of line (C) extends beyond 50 
line (B). 

FIG. 12 graphically demonstrates the data flow into and 
out of code buffer 22. The T-axis (abscissa) represents time, 
while the D-axis (ordinate) represents data size for each 
page of data. Thus, the gradient (rise/run) represents the data 55 
flow rate of the subtitles into the code buffer. Graph (C) 
represents the data flow of the subtitle data. The vertical 
portions of graph (Q indicate a transfer of subtitle data from 
the code buffer when the display time stamp (PTS) is aligned 
with the synchronizing clock (SCR) generated internally by 60 
subtitle decoder 7. The horizontal portions of the graph (Q 
indicate the transfer of subtitle data into the code buffer. For 
example, at a time that the display time stamp (PTS) for page 
(SO) is received by the code buffer, the previous page of 
subtitle data is transferred from the code buffer and page 65 
(SO) is written into the code buffer. When another display 
time stamp (PTS) is received by the code buffer, the subtitle 
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data of page (SO) is transferred out of the code buffer and 
page (SI) is written in. Similarly, the remaining pages (S2), 
(S3) are written into and read out of the code buffer as 
indicated. 

To precisely time the reading of the subtitle data from the 
code buffer with the display of the video image, delay 
compensation must be performed to allow for delays within 
the subtitle decoder. This is especially important where an 
external memory is employed as the display memory 
because an external memory increases the delay factor. 
Delay compensation is achieved by controlling the timing of 
the decode start command from system controller 14. The 
system controller delays the decode start command by a time 
equal to the processing of a letter box picture (approximately 
one field) and a delay caused by video decoding at the 
instant the synchronizing clock of the controller (SCR) is 
aligned with the display time stamp (PTS). Delay compen- 
sation is particularly useful, since the video, audio and 
subtitle data are multiplexed on the premise that the decode 
delay in each of the video, audio and subtitle data signals is 
zero in the data encoding apparatus. 

When the subtitle data for one page is read out of the 
display memory 22-2 (FIG. 13), the headers of the bit 
streams are separated therefrom by a parser 22-3 and the 
remaining data is forwarded to the inverse variable-length 
coder or run-length decoder 23, 24 during a vertical blanking 
period (V). Inverse VLC (Variable Length Coding) circuit 
23 (FIG. 2) subjects the subtitle data to variable length 
decoding. The variable length decoded subtitle data is com- 
posed of level data ("1" or "0") and run data as paired data. 
In the case where variable length decoding is not employed, 
the inverse VLC circuit may be bypassed and the subtitle 
data read from the code buffer will be directly output to 
inverse run-length circuit 24. Inverse run-length circuit 24 
conducts run-length decoding by generating the level of data 
from the number of run data elements. Thus, VLC circuit 23 
and run-length circuit 24 decompress the subtitle data which 
had been stored as compressed data in code buffer 22. 

The decompressed subtitle data is then sent to a 3:4 filter 
25. The 3:4 filter receives an xsqueeze signal from the 
system controller 14 indicating the aspect ratio of the 
corresponding television monitor Where the signal indicates 
that the monitor has a 4:3 aspect ratio, the 3:4 filter applies 
3:4 filtration processes to the subtitle data to match the size 
of the subtitles to the size of the video picture. In the 
preferred embodiment, the controller 35 reads 90 pixels 
worth of subtitle data from the code buffer 22 before the H 
sync pulse is generated. In the case where the television 
monitor already has a 16:9 aspect ratio, or the decompressed 
subtitle data represents fonts, the 3:4 filter is bypassed. 

A color look-up table 26 (CLUT) receives the subtitle data 
from the 3:4 filter 25 and the CLUT_data from the code 
buffer 22. The color look up table generates a suitable color 
from the CLUT_data for the subtitle data. The color look up 
table selects an address corresponding to the subtitle data for 
each pixel and forwards a mixing ratio K and color com- 
ponents Y (luminance), C^ (color difference signal R-Y) and 
C B (color difference signal B-Y) to the mixer 34. The color 
components Y, C x and C B> when mixed by mixer 34, at the 
mixing ratio K create a pixel with the color indicated by the 
color look up table. 

Background video data is incorporated in the arrangement 
of the color look-up table. For example, address 0 of the 
look-up table includes key data K having the value of 00 h; 
which means that the subtitle data will not be seen and the 
background video data will manifest, as shown by regions 
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Tl and T5 in FIG. 7c. Addresses lh to 6h of the look-up to a viewer a current event being displayed or speech being 

table include values of the key data K which increase uttered during the search. The subtitle to be decoded, and 

linearly (20, 40 ... CO hexadecimal); which means that the eventually displayed, will be different for the indicated scene 

subtitle pixels according to these addresses are mixed with or sound searched. For example, the subtitle decoded may be 

the background data as shown by the regions T2 and T4 in 5 * caption of the current event or a verse of speech. The 

FIG. 7c. Finally, addresses 8h to Fh of the look-up table P*™? 01 invention determines which subtitles are decoded by 

include values of key data K of EOh; which means that the dishngujshing between the types of subbdes and decoding 

components Y, Cr and Cb are mixed without any background ^ V"**** t0 * C tyi f °f **** * 1 

. , ^ , , . . _ . i tT^_, , appreciated that the other subtitles which do Dot relate to the 

video data as shown by region 13 in FIG. 7c. The co or ^ ^ ^ decoded ^ 

look-up table data is generated from tie system controller to * ^ ^ for ^ ^ ^ ^ 

and is previously downloaded to the CLUT " circuit before 

decoding. With the color look-up table, the filtered subtitle _ & . r . ... . 

. 4 . * , • . i~ „»i The present invention distinguishes between different 

data is transformed into the appropriate color pixel for *\ * - 

, *u * i • * w subtitles by categorizing the subtitles into a multiple of 

display an the television monitor. 4 , / ^ / . -j j* j . »- 4 i : \ 

r 3 stream data types (such as video, audio and subtitle types). 

FIG. 8 shows one example of a color look-up table where « M dcscribcd ^ reference to word detector 20 (FIG. 2), the 

the components Y, Cr, Cb and K are arranged according to videQ ^ fe st0fed on a record medium> such as disk 91, and 

the addresses 0 . . . F (hexadecimal). As will be explained, ^refcre reproduced therefrom, as streams of data, 

color wiping is performed by changing the <XUT_data, Conceptually, these multiple types of data streams of data 

thereby replacing part of the color look up table by the color are showQ m nGS 4A> B ^ ^ video streams mdicaled 

wiping color look up table shown m FIG. 11. Normally, a ™ by V_J, V_P, V_B, the audio streams by A, and the subtitle 

particular subtitle frame is refreshed several times because streams by Sw, Sp. The multiple types of data streams may 

frames are refreshed in a television signal several times a be MiUtnd on me disk and are reassembled by the decoding 

second. When the subtitles are refreshed, the same subtitle apparatus (no. 1) into coherent streams of information for 

data will be employed. However, the color will be different television display 

due to the changed color look up table. Thus the subtitles 25 ^ ^ ^ ^ ^ ^ ^ streams of ^ 

will appear to be color wiped as they are refreshed with each ^ ^ showQ . q nGS ^ B ^ dcpcnding upon 

consecutive frame. mc plavback mode ^ ^ shown by me subtitles Sp and Sw 

A mixer 34 (FIG. 2) mixes the pixels from the color WQere are norm al and trick mode subtitles, respectively. As 

look-up table 26 with video data from video decoder 3 (FIG. ^b the video and audio data streams, the subtitle data 

1). The resulting mixed data represents a video picture with streams may be scattered on the disk and, as shown in FIGS, 

superimposed subtitles and is ready to be output to a 4A, B, the subtitles are divided into three streams (Sp, Sw). 

television monitor. Mixer 34 is controlled to position the ^ tbKQ str eams are reassembled into one subtitle (Sp, Sw) 

subtitles within the video picture by referencing a in a forward playback mode by causing the pickup to jump 

u^ositioo signal generated by system controller 14 from over intermediary audio and video streams according to the 

commands of an operator via controller 35. The u_position arrows pointing from the left to the right in the figures (the 

value designates the vertical position for display on the subtitles in reverse playback are reassembled by following 

screen and may be varied (either by a user, the transmitter, ^ arrows f rom right to left). 

or otherwise) allowing a user to place the subtitles anywhere The decoding apparatus decodes only the video streams 

along a vertical axis. ^ y_I during trick mode reproduction because the video 

The decoding apparatus of the present invention may be streams V_J comprise the video frames corresponding to 

practiced with the parameters for the different signals shown the I-frame in a series of MPEG (Motion Pictures Experts 

in FIG. 6. However, the present invention is not limited to Group) frames which are not compressed on the basis of 

the parameters set forth in that figure and may be employed adjacent frames. On the other hand, the V_P and V_B 

in different video systems. 45 video streams correspond to predictively encoded frames 

With the present invention, a user has control over the which are compressed on the basis of adjacent frames and 

display of the subtitle through a mode display device 9 (FIG. decompression would require a longer period of time than 

1). System controller 14, upon command from the user, allowed during fast-forward/reverse of a trick mode, 

sends a control signal to mixer 34 (FIG. 2), turning the It will be appreciated that some video images do not have 

subtitles on or off. Since the present invention decodes 50 corresponding subtitle data because these video images are, 

subtitles in real time, the user does not experience any for example, redundant or have little action. To reproduce 

unpleasant delay when turning the subtitles on or off. In these video images during the trick mode would cause a 

addition, the subtitles can be controlled, by the user or viewer to watch uninteresting images. In the present 

otherwise, to fade-in/fade -out at a variable rate. This is invention, only the V_J streams have adjacent subtitle 

achieved by multiplying a fade coefficient to the pattern data 5S stream data Sw and are decoded/displayed so that a viewer 

representing the subtitles at a designated speed. This fuoc- does not have to wait through video frames which have little 

lion also allows an editor of the subtitles to present viewers interest. That is, only those frames which have significant 

with different sensations according to the broadcast of the activity, as predetermined upon encoding, will be assigned a 

audio/video picture. For example, news information may be subtitle and decoded for display. For example, a board room 

"flashed" rapidly to draw the attention of the viewer, 60 meeting scene where tittle activity takes place will be 

whereas subtitles in a slow music video "softiy" appear in bypassed until an event of interest occurs and a subtitle will 

order not to detract from the enjoyment of the music video. appear to alert the viewer of the interesting event The 

viewer, then, commands the system controller to leave trick 

Decoding Multiple Types of Subtitles mode ^ eQtcr normal p i ay back mode, whereupon the trick 

The data decoding apparatus of FIG. 1 is designed to 65 mode subtitles indicating the events no longer are displayed, 

search a specific scene in the video image or sound. To that Any scene in the video image or audible speech can be 

end, the present invention decodes subtitles which indicate searched. To that end, the present invention distinguishes 
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between multiple types (such as video, audio and subtitle 
types) of data streams (such as video, audio or subtitle data). 
Thus, for example, when a scene is searched, the system 
controller causes the subtitle decoder to decode only the 
subtitle data streams describing the video data. Similarly, 
when a sound is to be located, the system controller causes 
the subtitle decoder to decode only the subtitle data streams 
describing the audio data. The system controller operates by 
determining where the streams of the multiple types of data 
streams are located on the disk and causing the respective 

data streams for the type searched to be reproduced and sent Sys(em 14 (F1G 2) acls „ a and 

to the decoder. executes the above operation, causing the word detector to 

In a first embodiment of the invention, the system con- determine if theuser_data_flag is set to "1" and, if so, treats 

troUer determines where the streams are on the disk by thenextl6bitsaslength_of_user_data;thenext32bitsas 

extracting a tabic of contents (TOQ from the beginning of next_subtitle_address_offset; the next 8 bits as reserved; 

the disk which provides the addresses for the multiple types * c next 24 bits as prcviou^ubhde adaress_offset; and 

of data streams. FIGS. 3A, B depict a table of contents last 8 blts as "served. The word detector forwards tas 

,. t . . t . . , , r ^ ntf > ntc information to the system controller, via the controller 35, 

according to the present mvention. The table ot contents 2 o , . . i ■ u* »t tl . 

, 6 i\ . , e r i_ and continues detecting subtitle streams. The system con- 

shown .n FIG. 3A identifies frames by frame number ^ ^ ^ ^ 

(subcode frame #) and pointers (POINT) pointing to a track ^ deco& apparatus ^ described. 

on the disk at which the corresponding frame is located, and 

time codes (PMIN, PSEC, PFRAME) corresponding to that Encoding Technique 

frame. The table of contents of FIG. 3B identifies stream 25 The encoding technique employed in the present inven- 

data according to the multiple types of streams by indicating tion will be described in more particular detail with refer- 

the frame, starting sector address, and ending sector address ence to FIGS. 7 A, 7B, 7C and FIG. 8. As an example, the 

of recorded data streams arranged by type (i.e., video, audio technique for encoding the letter "A" of FIG. 7A will be 

or subtitle). From this table of contents, the system control- explained. The letter "A" is scanned along successive hori- 

ler can determine all of the streams in a particular type of 30 zontal lines and the fill data of FIG. 7B is generated for the 

stream data and cause drive control 15 (FIG. 1) to jump the letter "A" along each horizontal line. It will be noted that the 

pickup to the sector indicated by the start_sector address in l evcl "E^ demarks the highest level for recreating a color 

the table of contents p ^ ^ e co ^ or ^ 0 °^" U P ta ^ e shown m FIG. 6, whereas 

level "0" represents a lack of subtitle data. 

In a second embodiment of the invention, the sector 35 xhe key data (K) (or mix ing ratio) determines the degree 

addresses for the multiple types of streams are collected in to which the fill data is mixed with background video, 

an area of the disk called a stream map. Unlike the table of Regions Tl and T5 of the key data correspond to areas in the 

contents, the stream map is not necessarily confined to the video picture that are not superimposed with the fill data; 

beginning of the disk, but may be located at any sector as a therefore, these areas are designated as level 0 as indicated 

packet. The stream map is, thus, arranged as a packet with 40 by address 0 in FIG. 8. Regions T2 and T4 are mixed areas 

video, audio, subtitle, blanking, packet length of stream, where the subtitles are gradually mixed with the background 

identifier, and length of stream map information. The system video picture so that the subtitles blend into the background 

controller references the stream map in a similar manner to video picture and do not abruptly contrast therewith. Any of 

the table of contents, thus causing the pickup to reproduce ^ fill data in this area is stored in addresses 1 through 6 of 

subtitle streams corresponding to the type of data stream to 45 ^ color look " u P tablc - ^ main P orUon of thc letter "A" 

be searched, for example, sending these reproduced streams is displayed witmn the T3 region where the background 

t • 4 *-i j information is muted. The subtitle information in region 13 

to the subtitle decoder. . , _ ^ _ , . t 

is stored as addresses 7 to F (hexadecimal). The color 

In a third embodiment, the sector addresses of the previ- look-up table of FIG. 8 is arranged in varying degrees of the 

ous and following streams of a currently reproduced subtitle 50 luminance component Y. When a pixel in the region T3 is to 

stream are stored in each subtitle stream. Since the sector be stored, for example, and the level of the luminance 

addresses are in the subtitle streams, the sector addresses are component Y for that particular pixel is 20 (hexadecimal), 

decoded and word detector 20 of the subtitle decoder (FIG. the color information for that pixel is obtained from address 

2) detects the subtitle stream sector addresses (subtitle 9 (FIG. 8). In this manner, the remaining pixels for the 

stream sector address) and forwards them to the system 55 subtitle characters are encoded, 

controller, via controller 35. As each subtitle is decoded in, Encoding Apparatus 

for example, the forward playback mode, the system con- ^ eocoding apparatus of lhc present invention is 

trailer recalls the following sector address for the next dep i cte a in FIGS. 9A, B. Audio and video information is 

subtitle and causes the pickup to jump to the sector indicated ^ received by a microphone 53 and video camera 51, 

by that following address to reproduce the next subtitle. respectively, and forwarded to a multiplexer 58. The subtitle 

Similarly, in the reverse playback mode, the system con- data are entered through either a character generator 55 or a 
trailer recalls the previous sector address for the previous flying spot scanner 56 and encoded by a subtitle encoding 

subtitle and causes that subtitle to be reproduced. circuit 57. The encoded subtitle information is sent to 

Specifically, the word detector detects whether a stream 65 multiplexer 58 and combined with the audio/video informa- 
includes sector addresses according to the following opera- tion for recording onto a record disc 91 or supplied to a 

tion: channel for transmission, display, recording or the like. 
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Video camera 51 generates the video signal and supplies 
the same to a video encoding unit 52 which converts the 
video signal from analog to digital form. The digitized video 
signal is then compressed for video transmission and for- 
warded to a rate controller 52a, which controls the rate that 5 
the compressed video data is transferred to the multiplexer 
in synchronism with the rate that the subtitles are sent to the 
multiplexer. In this manner, the compressed video data is 
combined with the subtitle data at the correct time. Similarly, 
audio information is obtained by microphone 53 and 
encoded by an audio encoding unit 54 before being sent to 
the multiplexer. The audio encoding unit does not necessar- 
ily include a rate controller because the audio data may 
ultimately be recorded on a different track or transmitted 
over a different channel from the video data. 5 

The subtides are generated by either character generator 
55 or flying spot scanner 56. The character generator 
includes a monitor and a keyboard which allows an operator 
to manually insert subtitles into a video picture. The operator 
edits the subtitles by typing the subtitles through the key- 20 
board. Flying spot scanner 56, on the other hand, is used for 
the situation where subtitles are already provided in an 
external video picture or scanned in as text. The flying spot 
scanner scans the video picture and determines where the 
subtitles are positioned and generates corresponding subtitle 25 
data therefrom. The subtitles from the flying spot scanner are 
pre-processed by a processing circuit 63 to conform with 
subtitles generated by the character generator before further 
processing by the subtitle encoding circuit. 

The subtide data from either character generator 55 or 30 
flying spot scanner 56 are, then, selected for compression. 
The character generator outputs blanking data, subtitle data 
and key data. The subtitle data and key data are forwarded 
to a switch 61 which is switched according to a predeter- 
mined timing to select either the subtitle or key data. The 3S 
selected data from switch 61 is filtered by a filter 72 and 
supplied to another switch 62. Switch 62 switches between 
blanking data, the filtered data from the character generator, 
and the processed data from the flying spot scanner. When 
it is determined that no subtitles are present, the blanking 40 
data is chosen by switch 62. Where subtitles are present, 
switch 62 chooses between the character generator data or 
the flying spot scanner data, depending upon which device 
is being used to generate the subtide data. 

The data selected by switch 62 is quantized by a quanti- 45 
zation circuit 64, using a quantization level based on data fed 
back from a subtitle buffering verifier 68. The quantized 
data, which may be compressed, is supplied to a switch 69 
and (during normal operation) forwarded to a differential 
pulse code modulation (DPCM) circuit 65 for pulse code 50 
modulation. The modulated data is run-length encoded by a 
run-length coding circuit 66, variable-length encoded by a 
variable -length encoding circuit 67 and forwarded to the 
subtide buffering verifier 68 for final processing before 
being sent to multiplexer 58. 55 

Subtitle buffering verifier 68 verifies that the buffer is 
sufficiently filled with data without overflowing. This is 
done by feeding a control signal (referred to in FIG. 9 A as 
a filter signal) back to the quantization circuit. The control 
signal changes the quantization level of the quantization 60 
circuit, thereby changing the amount of data encoded for a 
particular subtide. By increasing the quantization level, the 
amount of data required for the subtide data is reduced and 
the bit rate of data flowing to the subtitle buffering verifier 
is consequently reduced. When the subtide buffering verifier 65 
determines that there is an underflow of data, the control 
signal decreases the quantization level and the amount of 
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data output from the quantization circuit increases, thereby 
filling the subtide buffering verifier. 

The subtitle buffering verifier is also responsible for 
preparing the subtide data for transmission (over television 
airwaves, for example). The subtide buffering verifier, to this 
end, inserts information necessary to decode the encoded 
subtide data. This information includes a normal/special 
play signal which indicates whether the subtides are 
recorded in a normal or special (fast-forward/reverse) mode 
(referred to above as the trick mode). An upper limit value 
signal is inserted which indicates the upper limit for the 
memory size of the subtide data for a frame. An EOP signal 
marks the end of page for the subtitle data frame and also is 
inserted. A time code signal is inserted which is used as the 
time stamp PTS in decoding. Subtitle encoding information 
is inserted and includes information used in encoding the 
subtide data, such as the quantization factor. Positional 
information is inserted and is used as the position_data 
upon decoding. A static/dynamic signal is inserted which 
indicates whether the subtitle data is in static or dynamic 
mode. The subtide buffering verifier also inserts the color 
look up table address for transmission to the decoder so that 
the colors of the display will match the colors employed in 
creating the subtides. 

The subtitle buffering verifier is preferably a code buffer 
similar to the code buffer of the decoder (FIG. 2). To that 
end, it is useful to think of the operation of the subtide 
buffering verifier to be in symmetry (i.e., performing the 
inverse functions of the code buffer) with the code buffer. 
For example, the color pixels of the subtides are converted 
into digital representations; the resultant digital subtitles are 
encoded by the run length encoder and the variable length 
encoder; header information is added; and the resultant 
subtide information is stored in a buffer and forwarded to 
multiplexer 58 for multiplexing with the audio and video 
data. 

Multiplexer 58 preferably employs time-sliced multiplex- 
ing; and also provides error correction processing (e.g., error 
correction coding) and modulation processing (e.g., EFM, 
eight-to-fourteen modulation). The multiplexed data is then 
transmitted (via television broadcasting, recording, or other 
means of transference) to the decoding apparatus for decod- 
ing and display. 

Encoding Multiple Types of Subtitles 

The present invention permits a viewer to search for a 
specific scene or audible speech by distinguishing between 
multiple types of data streams and selecting those data 
streams for display which are of the type relating to the 
search. For example, a video image is located by displaying 
the data streams of the subtides describing the video image. 
The manner in which the encoder, already described with 
reference to FIGS. 9A and B, encodes the multiple types of 
data streams will now be discussed. After multiplexer 58 
multiplexes the audio, video and subtitle data streams, the 
multiplexed data is sent to a sectorizing processor 100 which 
arranges the data streams into fixed length sectors of pack- 
ets. At this point, the data streams are ready for airwave 
transmission. When the data streams are to be recorded on 
a disk, however, a table of contents (TOC) & stream map 
generator 101 determines the addresses of the data streams 
to be recorded on the disk. 

According to the first embodiment, the TOC & stream 
map generator generates the table of contents shown in 
FIGS. 3A, B from the sectors generated by the sectorizing 
processor and the video/audio search information generated, 
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for example, by a viewer. In the second embodiment, the 
TOC & stream map generator generates the stream map 
from the sectors generated by the sectorizing processor. 
Unlike the previous embodiment, the TOC & stream map 
generator inserts the stream map as a packet onto the disk. 5 
In the first two embodiments, the system controller 14 of the 
data reproducer (or receiver) reads the table of contents or 
the stream map directly and causes the decoding apparatus 
(FIG. 1) to decode the streams which relate to the data type 
being searched. In the third embodiment, the TOC & stream 10 
map generator inserts the stream addresses for the previous 
and following addresses into each of the subtitle streams. 
Unlike the first two embodiments, the system controller 
must cause the subtitle decoder to decode the subtitle stream 
and extract therefrom the sector addresses. As described in is 
regard to decoding multiple types of streams, the TOC & 
stream map generator encodes each stream with 1 bit of a 
user_data_flag that indicates whether stream addresses are 
forthcoming in the stream; the next 16 bits as length__of_ 
user_data; the next 32 bits as next_subtitle_address_ 20 
offset; the next 8 bits as reserved; the next 24 bits as 
previous_subtitle„address_oflket; and the last 8 bits as 
reserved. 

According to the present invention, the video image, the 
audio track, and the subtitles are arranged in units of frames 25 
on the disk and the system controller accesses information 
by recalling from the disk the streams making up these 
frames. With this scheme, the system controller can cause 
the decoder to decode only those types subtitles which relate 
to the particular scene or sound to be searched. Thus, a 30 
viewer can browse through the video picture by reading the 
subtitles and view only those portions which are of interest. 
For that matter, the order or speed at which the information 
is reproduced can be altered freely. For example, if the 
viewer has auditory problems or is teaming the language 35 
spoken on the audio track, the system controller can, under 
operator control, slow the rate of reproduction for the audio 
track to allow the viewer to understand the reproduced 
speech. Another application of the present invention relates 
to color wiping, where the information of color wiping 40 
changes frequently between frames. A viewer desiring to 
sing a portion of a Karaoke piece, for example, may search 
for a particular point in the song by viewing the subtitles and 
halting the search at the instant when the portion of the 
subtitles to be sung is highlighted by the colorwiping. 4 * 

Colorwiping Encoding 

Colorwiping refers to a process by which an image, such 
as the subtitles, is gradually overlayed with another image. 
An exemplary application of colorwiping is highlighting, so 
wherein a frame of subtitles is dynamically highlighted from 
left to right with the passage of time. The present invention 
performs colorwiping by changing the color look up table at 
different points of time during the subtitle display. For 
example, an initial subtitle frame is generated with the ss 
standard color look up table in FIG. 8. When colorwiping is 
performed, the color look up table is changed to the color 
wiping look up table of FIG. 11. With the passage of each 
frame, the gradual change of the position at which the color 
look up table is changed from the colorwiping to the so 
standard color look provides the sensation that the subtitles 
are changing color dynamically over time from left to right. 

An encoding operation for color wiping will now be 
discussed with reference to FIGS. 9 A and 10. During the 
course of encoding subtitles, an operator may desire to color 65 
wipe the previously encoded subtitles. To that end, the 
operator is provided with a wipe lever 81 to control the 
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colorwiping and a monitor 84 to view the color wiping in 
real time. The wipe lever is connected to an adapter 82 to 
adapt the analog voltages of the wipe lever to digital signals 
suitable for digital manipulation. The digital output of the 
adapter is fed to both a switcher 83 and a wipe data sampler 
70. The switcher switches the color look up table to values 
represented by the position of the wipe lever and generates 
color pixels of the subtitles for display on the monitor. Thus, 
the operator can visually inspect the colorwiping procedure 
while it occurs and adjust the speed or color of the wiping 
to satisfaction. 

The wipe data sampler and position sampler 70 deter- 
mines from the adapter signals where in the video picture the 
color look up table is to be changed and outputs this 
information to encoding circuits 65, 66 and 67 (via switch 
69) for encoding and transmission to multiplexer 58. FIGS. 
10A and 10B depict a block diagram of the operation of the 
wipe data and position sampler. A comparator 301 compares 
a present pixel signal generated by the adapter with a 
previous pixel signal from the adapter. This is achieved by 
transmitting the present pixel value to input A of comparator 
301 while supplying the previous pixel value latched in a 
register 300 to input B of comparator 301. The comparator 
outputs a boolean "true" value to a counter 302 (which is 
reset at every horizontal or vertical sync pulse) when the 
present and previous pixels have the same value and, in 
response thereto, the counter increments a count value. That 
is, the comparator registers a true condition when the pixels 
up until that point are generated from the same color look up 
table. At the point where the color look up table changes, 
therefore, the present and previous pixels become unequal 
(i.e., their color changes) and the comparator generates a 
"false" boolean condition. The count value, thus, is equal to 
the number of matches between the present and previous 
values, which is the same as the position at which the color 
look up table changes. The count value is latched by a 
register 303 upon the following vertical sync pulse and 
transferred to the encoding circuits (via switch 69) for 
transmission. 

Colorwiping Decoding 

Color wiping decoding will now be discussed with ref- 
erence to FIGS. 14A-C and 15. FIG. 14A shows the position 
here the color look up table is switched at point A from a 
color wiping look up table (FIG. 11) to the standard color 
look up table (FIG. 8). FIG. 14B depicts a pattern of subtitle 
and colorwipe data arranged in discrete blocks of presenta- 
tion time stamps (PTS(n) . . . PTS(n+t)). The first presen- 
tation time stamp PTS(n) corresponds to normal subtitle data 
and the remaining presentation time stamps PTS(n+l . . . 
n+t) correspond to colorwiping data (WPA . . . WPZ). FIG. 
14C shows successive frames (n . . . n+t) which correspond 
to the presentation time stamps. To execute colorwiping, 
each successive colorwiping frame (WPA . . . WPZ) sets the 
point where the color look up table is switched (point A) 
further along the displayed subtitle, thereby dynamically 
performing colorwiping as a function of time. 

An operational block diagram of the colorwiping decod- 
ing is depicted in FIG. 15. The vertical sync pulse triggers 
a register 205 to latch the current subtitle frame from a 
display buffer (FIG. 15 shows a colorwiping frame WP 
being latched). The colorwiping data latched by the register 
indicates the position of the color look up table switching. A 
pixel counter 208 decrements the value indicated by the 
colorwiping data at each horizontal sync pulse and outputs 
a boolean "true" flag to color look up table 26. While the flag 
is "true" the color look up table employs the colorwiping 
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table (FIG. 11) to decode the colors of the subtitle pixels. 
When the pixel counter reaches zero, the position of color 
look up table switching is reached and the pixel counter 
issues a boolean "false" flag to color look up table 26. At this 
time, the color look up table switches the colorwiping color 
look up table (FIG. U) to the standard look up table (FIG. 
8), and the remainder of the subtitle frame is displayed in 
standard color mode. Each successive colorwiping frame 
(WPA . . . WPZ) moves the position of switching; thus, each 
refreshed subtitle frame advances (or retreats) the 
colorwiping, thus performing dynamic colorwiping. 

The colorwiping color look up table in FIG. 11 incorpo- 
rates two sets of colors (one set for addresses Oh to 7h and 
a second set for addresses 8h to Fh). Thus, the colorwiping 
color can be changed to a secondary color simply by 
changing the most significant bit (MSB) of the color look up 
table address. For example, the first set of colorwiping 
colors has a MSB of "0", while the second set has a MSB 
of "1". Changing the MSB of address 7h to a "1" transforms 
the address to Fh and the colorwiping color changes. This 
may be done, for example, by setting the MSB equal to the 
flag of pixel counter 208. 

Employing the MSB to change between color sets has the 
advantage of reducing the number of bits required to be 
encoded. Since the MSB is known, only the three lower 
order bits need to be encoded where 4 bits are employed for 
every pixel. Where two bits are employed for every pixel, 
the subtitle data is coded only for the least significant bit. In 
a 4 bits per 2 pixel format, only the MSB is employed fbr 
color control and the remaining three bits can be reserved for 
pixel information. Thus, by using the MSB the number of 
bits encoded can be decreased and the overall processing 
time for encoding and decoding is optimized. 

Dynamic Subtitle Positioning 

The subtitles are repositioned dynamically, i.e., as a 
function of time, by employing a similar technique as 
described above with reference to colorwiping. As shown in 
FIGS. 16A-C and 17 the position data is measured along the 
horizontal axis (FIG. 16A) and is transferred to the subtitle 
decoder with the subtitle data during the appropriate frame 
(FIG. 16Q corresponding to a presentation time stamp 
(PTS(n), for example; FIG. 16B). 

The positioning operation will now be explained with 
reference to FIG. 17. The position data is a value represent- 
ing the position of the subtitle frame along the horizontal 
axis and is read out from the display buffer and latched by 
register 205 on each vertical sync pulse. Pixel counter 208 
decrements the position data on each horizontal sync pulse 
and send a boolean flag to controller 35 (FIG. 2) to indicate 
that the position of the subtitle frame has not been reached. 
When the pixel counter reaches zero, the position of the 
subtitle frame has been reached and the boolean flag is 
toggled to indicate this to the controller. The controller, 
which has been delaying the reading operation of code buffer 
22 (FIG. 2), then causes the code buffer to read out the 
subtitle data to run length decoder 24 (FIG. 2). The subtitle 
data is then decoded as described above and displayed with 
the corresponding video image. In this manner, the position 
of the subtitle frame is changed with each frame; thus 
providing dynamic movement of the subtitle frame. 

It will be appreciated that the present invention is appli- 
cable to other applications, such as television or video 
graphics. It is, therefore, to be understood that, within the 
scope of the appended claims, the invention may be prac- 
ticed otherwise than as specifically described herein. 

What is claimed is: 

1. An encoding apparatus for encoding data streams of 
multiple types including video, audio and subtitle types of 
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data streams, onto a record medium, a sequence of the data 
streams for each type constituting a respective video, audio 
and subtitle signal which combine into a video broadcast, 
said encoding apparatus comprising: 

encoding means for encoding said data streams of mul- 
tiple types to be recorded on the record medium and 
recording said data streams on the record medium; 
address generating means for generating addresses at least 
for subtitle data streams pointing to a position where a 
respective subtitle data stream is to be recorded on the 
record medium; and 
address storing means for storing said addresses generated 
by said address generating means in a predetermined 
area of the record medium, 
wherein an event represented by video or audio data in the 
data streams can be located by searching the subtitle 
data streams associated with the respective video or 
audio data to be located. 

2. The encoding apparatus of claim 1, wherein subtitles 
are generated from the recorded data streams to be displayed 
exclusively during a trick data playback mode, 

wherein said encoding means encodes said subtitles 
exclusively generated for trick mode as data streams on 
said record medium, 

wherein said address generating means generates 
addresses for said subtitles exclusively generated for 
trick mode, and 

wherein said address storing means stores the addresses of 
the subtitles exclusively generated for trick mode at a 
position on the record medium indicating that the 
addresses are of the subtitle type exclusively generated 
for trick mode such that said event represented by data 
can be located by accessing said data streams of said 
subtitles exclusively generated for trick mode. 

3. The encoding apparatus of claim 2, wherein the address 
storing means further comprise table of contents means for 
storing the addresses in a table according to the type of 
recorded data stream in an area of the record medium 
preceding an area where the data streams are stored. 

4. The encoding apparatus of claim 2, wherein the address 
storing means further comprise stream map means for 
storing the addresses according to the type of recorded data 
stream in a packet of data streams recorded on the recording 
medium. 

5. The encoding apparatus of claim 2, wherein the address 
storing means further comprise means for storing, in each of 
the data streams of the subtitles exclusively generated for 
trick mode, a next address of the following data stream and 
a previous address of the preceding data stream in the 
sequence for the data stream of subtitles. 

6. An encoding method for encoding data streams of 
multiple types, including video, audio and subtitle types of 
data streams, onto a record medium, a sequence of the data 
streams for each type constituting a respective video, audio 
and subtitle signal which combine into a video broadcast, 
said encoding apparatus comprising: 

encoding said data streams of multiple types to be 
recorded on the record medium and recording said data 
streams on the record medium; 

generating addresses at least for subtitle data streams 
pointing to a position where a respective sub tide data 
stream is to be recorded on the record medium; and 

storing said addresses in a predetermined area of the 
record medium, 

wherein an event represented by video or audio data in the 
data streams can be located by searching the subtitle 
data streams associated with the respective video or 
audio data to be located. 

1. The encoding method of claim 6, wherein subtitles are 
generated from the recorded data streams to be displayed 
exclusively during a trick mode; and further comprising the 
steps of: 
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encoding said subtitles exclusively generated for trick 
mode as data streams on said record medium, 

generating addresses for said subtitles exclusively gener- 
ated for trick mode, and 

storing the addresses of the subtitles exclusively gener- 
ated for trick mode at a position on the record medium 
indicating that the addresses are of the subtitle type 
exclusively generated for trick mode such that said 
event represented by data can be located by accessing 
said data streams of said subtitles exclusively generated 
for trick mode. 

8. The encoding method of claim 7, wherein the step of 
storing the addresses of the subtitles exclusively generated 
for trick mode comprises storing the addresses in a table of 
contents according to the type of recorded data stream into 
an area of the record medium preceding an area where the 
data streams are stored. 

9. The encoding method of claim 7, wherein the step of 
storing the addresses of the subtitles exclusively generated 
for trick mode further comprises the step of grouping the 
addresses according to the type of data stream in a packet of 
data streams on the recording medium. 

10. The encoding method of claim 7, wherein the step of 
storing the addresses of the subtitles exclusively generated 
for trick mode further comprises the step of storing, in each 
of the data streams of the subtitles exclusively generated for 
trick mode, a next address of the following data stream and 
a previous address of the preceding data stream in the 
sequence for the data stream of subtitles. 

11. A recording medium for recording thereon data 
streams of multiple types including video, audio and subtitle 
types of data streams, a sequence of the data streams for each 
type constituting a respective video, audio and subtitle 
signal which combine into a video broadcast, said data 
streams being prepared by the steps of: 

encoding said data streams of multiple types to be 
recorded to said recording medium and recording said 
data streams on the record medium; 

generating addresses at least for subtitle data streams 
pointing to a position where a respective subtitle data 
stream is to be recorded to said recording medium; and 

storing said addresses in a predetermined area of said 
recording medium, wherein an event represented by 
video or audio data in the data streams can be located 
by searching the subtitle data streams associated with 
the respective video or audio data to be located, 

12. The recording medium according to claim 11, wherein 
said addresses are stored in a table arranged by the type of 
data. 

13. The recording medium according to claim 11, wherein 
said addresses are stored by the type of data stream in a 
packet on the recording medium. 

14. The recording medium according to claim 11, wherein 
in each of the data streams of the subtitles exclusively 
displayed during a search, a next address of the following 
data stream and a previous address of the preceding data 
stream are stored in the sequence for the data stream of 
subtitles. 

15. A decoding apparatus for decoding data streams of 
multiple types, including video, audio and subtitle types of 
data streams reproduced from a record medium, a sequence 
of the data streams, for each type constituting a respective 
video, audio and subtitle signal which are combined into a 
video broadcast, said decoding apparatus comprising: 
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decoding means for decoding the multiple types of data 
streams which were recorded on the record medium 
reproduced from the record medium; and 
control means for controlling said decoding means to 
5 decode data streams of the subtitle type which are to be 
displayed exclusively during a search, such that any 
event represented by video or audio data in the data 
streams is beatable by searching the subtitle data 
streams associated with the respective video or audio 
data to be located, 
to 16. The decoding apparatus of claim 15 wherein the 
control means comprises means for reproducing a table of 
contents recorded on the record medium at a position 
preceding the data streams recorded thereon, said table of 
contents including addresses of the data streams of the 
15 subtitle data type which are to be displayed exclusively 
during said search. 

17. The decoding apparatus of claim 15 wherein the 
control means comprises means for reproducing a stream 
map from a packet of the data streams recorded on the record 
medium, said stream map including addresses of the data 

20 streams of the subtitle data type which are to be displayed 
exclusively during said search. 

18. The decoding apparatus of claim 15 wherein the 
decoding means decodes data streams reproduced from 
addresses of the record medium and comprises means for 
reproducing from the record medium a next address of the 

25 following data stream and a preceding address of the pre- 
vious data stream of the sequence in each data stream is to 
be displayed exclusively during said search. 

19. A decoding method for decoding data streams of 
multiple types, including video, audio and subtitle types of 

30 data streams, reproduced from a record medium, a sequence 
of the data streams for each type constituting a respective 
video, audio and subtitle signal which are combined into a 
video broadcast, said decoding method comprising the steps 
of: 

decoding the multiple types of data streams which were 
35 recorded on the record medium and reproduced from 
the record medium; and 
controlling said decoding to decode data streams of the 
subtitle type which are to be displayed exclusively 
during a search, such that any event represented by 
40 video or audio data in the data streams is locatable by 
searching the subtitle data streams associated with the 
respective video or audio data to be located. 

20. The method of claim 19 wherein the controlling step 
comprises the step of reproducing a table of contents 

45 recorded on the record medium at a position preceding the 
data streams recorded thereon, said table of contents includ- 
ing addresses of the data streams of the subtitle data type 
which are to be displayed exclusively during said search- 

21. The method of claim 19 wherein the controlling step 
comprises the step of reproducing a stream map from a 

50 packet of the data streams stored on the record medium, said 
stream map including addresses of the data streams of the 
subtitle data type which are to be displayed exclusively 
during said search. 

22. The method of claim 19 wherein the decoding step 
55 decodes data streams reproduced from addresses of the 

record medium and comprises the step of reproducing from 
the record medium a next address of the following data 
stream in the sequence and a preceding address of the 
previous data stream in the sequence in each data stream to 
60 be displayed exclusively during said search. 

***** 
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